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37 CFR 1.136(a). 
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Application Papers 
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DETAILED ACTION 

Response to Amendment 

1 . Applicant's arguments filed on 8/4/99 in Paper 6 with respect to claims 1 -22 have been 
fully considered but they are not persuasive. 

2. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless ~ 

(e) the invention was described in a patent granted on an application for patent by another filed in the United 
States before the invention thereof by the applicant for patent, or on an international application by another who 
has fulfilled the requirements of paragraphs (1), (2), and (4) of section 371(c) of this title before the invention 
thereof by the applicant for patent. 

3. Claims 1-22 are rejected under 35 U.S.C. 102(e) as being anticipated by Zhang et al., 
(hereinafter referred to as "Zhang"), as was set forth in the Office Action of 4/28/99 mailed as 
Paper 5. 

The Applicant presents six substantive arguments contending the Examiner's rejection of 
claims 1-22 under 35 U.S.C. 102(e) as being anticipated by Zhang et al., (hereinafter referred to 
as "Zhang"), as was set forth in the Office Action of 4/28/99 mailed as Paper 5. However, after a 
careful consideration of the arguments presented and further scrutiny of the Zhang reference, the 
Examiner must respectfully disagree for the reasons that follow below. 

Firstly, the Applicant argues that Zhang's three algorithms used for image analysis are not 
all directed towards key frame selection, but for the segmentation processes and shot breaks 
detections (Paper 6: page 4, lines 8-33; page 5, lines 1-12; page 7, lines 4-13). The Examiner 
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respectfully disagrees. In particular, the algorithms for selecting segmentation 
boundaries according to their established thresholds as discussed in the rejection (Zhang: column 
4, lines 62-68; column 5, lines 1-68; column 6, lines 1-63) are part of the initialization steps 
(Zhang: column 5, lines 15-30) in executing the video segmentation and shot 'break detection 
(Zhang: column 6, lines 35-63). It is noted that the key frame selection process also discusses an 
initialization step of the system parameters (Zhang: column 7, lines 30-40; figure 4, element 502). 
The Examiner asserts that this initialization step is established in terms of the three previously 
discussed algorithms for the purpose of key frame selection, and accordingly, are not entirely 
different processes but a modification of system parameters for three different desired results. It is 
further important to note that Zhang touts the "flexible" nature (Zhang: column 4, lines 20-26) of 
the multi-pass use of algorithm (Zhang: column 6, lines 30-35). Accordingly, the Examiner 
maintains that the Zhang staggered use of algorithms are directly implemented for the key frame 
selection process, as well. 

Secondly, the Applicant argues that Zhang fails to disclose the use of "structural difference 
measures" based on an incorrect assumption of Zhang's discussion of the art (Paper 6: page 5, 
lines 23-34; page 6, lines 1-4). The Examiner respectfully disagrees. The pertinent section of the 
reference (Zhang: column 7, lines 52-61). What that section states is that known prior art methods 
of determination of temporal variations or "structural difference measurements" heavily depending 
upon the tracing of the positions and sizes of the objects for key frame extraction, but this prior 
art method is not desired as it is being impractical because it relies upon accurate motion field 
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detection, and complicated image warping (Zhang: column 7, lines 54-59). However, Zhang 
clearly says that the current method of the patent doesn't use this prior art method, using content 
based temporal variation determinations based on the metrics as discussed above and executed in 
conjunction with the depicted processes of figures 4 & 4A (Zhang: column 7, lines 59-62). Lastly, 
while the metric based on histogram comparison does ignore spatial changes as the applicant has 
duly noted (Zhang: column 4, lines 1-5), the likelihood ratio is very robust in terms preserving 
"content" behavior in a frame especially being tied optimally to the segmentation process (Zhang: 
column 3, lines 60-68). In particular, the pixel histogram metrics (Zhang: column 4, lines 1-25) 
for each of the color components of the TV/video signal (Zhang: column 4, lines 55-60) would 
read upon the chromatic difference measurements, while the likelihood ratio difference metric 
would be used for the structural difference metric which would more suitable for tracking content 
variations on a frame specific region basis (Zhang: column 3, lines 49-53), and this is combination 
that would read upon the instant invention. 

Thirdly, while discussing the nature of the two claimed measurements, the Applicant goes 
on to discuss the "orthogonal" nature of two claimed difference measurements (Paper 6: page 6, 
lines 6-22) as a means for adding weight the limitations of pertaining to these measurements. 
However, in response to applicant's argument that the references fail to show certain features of 
applicant's invention, it is noted that the features upon which applicant relies (i.e., "orthogonal 
nature" ) are not recited in the rejected claim(s) 1-9, and 1 1-22. Although the claims are 
interpreted in light of the specification, limitations from the specification are not read into the 
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claims. See In re Van Geuns, 988 F.2d 1 181, 26 USPQ2d 1057 (Fed. Cir. 1993). Now, going to 
the "orthogonal nature" of claim 10, which the Applicant has accurately noted as an inadvertent 
oversight on the part of the prior Office Action (Paper 6: page 7, lines 14-19), the Examiner 
asserts that Zhang's disclosure also describes the "orthogonal nature" of the two measurements as 
they have been described above would appear to have been summarily met, since the Examiner 
has already asserted that the region dependent likelihood ratio metric would address the 
"structural properties" of the frames, while the color component histograms would address 
brightness, or distribution of color over the frames, and equal the presented definition the 
"orthogonal nature" of the first and second measurements. 

Fourthly, the Applicant argues that the chromatic difference measure and structure 
difference measure are user selectable, whereas the Examiner's application of the Zhang's 
difference metrics above would not have the thresholds as being user selectable (Paper 6: page 6, 
lines 23-38; page 7, lines 1-3). The Examiner respectfully disagrees. In particular, while an initial 
selection of parameter's thresholds might be characterized as "automatic" based on statistics of 
the video frame as is noted by the Applicant (Paper 6: page 6, line 26), the continued use of those 
parameters are "tunable" (Zhang: column 7, lines 20-30) and this is further claimed to selectable 
(Zhang: column 12, lines 45-65), which clearly reads on the claims. 

Furthermore, the Applicant argues that Zhang fails to meet the limitation that the second 
difference measure is more computationally intensive that the first difference measure and that 
they are executed in a hierarchial manner (Paper 6: page 7, lines 20-31). The Examiner 
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respectfully disagrees. Merely from a pictorial representation of the key frame selection process, 
one can see that the execution of step of 506 and then of step 510 (Zhang: figure 4, elements 
506, 510) occur in a hierarchial fashion. Both of these steps are difference metric dependent and 
proceed towards the determination of the key frame (Zhang: figure 4 A, element 512). 
Furthermore, it is noted that in making these determinations using a multi-pass process, the first 
difference metric execution is run at a lower resolution for faster processing than the second 
difference metric as it pertains to the process of figures 3A1-3A3 (Zhang: column 6, lines 20-35), 
and this strategy carries over to the process of key frame selection as depicted in figures 4-4B. In 
particular, the use of "looser" thresholds at lower levels or processing steps (Zhang: column 6, 
lines 47-56), will result in those steps being less computationally intensive than the high level steps 
(Zhang: column 7, lines 1-10). 

Lastly, with regards to using three difference metrics as specified (Paper 6: page 8, lines 1- 
7), the Examiner notes that this can also be met by the reference as well. Merely from a pictorial 
representation of the key frame selection process (Zhang: figure 5C, element 652) , one can see 
the execution of third hierarchically implemented difference metrics. The third and closest to the 
key frame selection process in question is step 650 (Zhang: figure 5C, element 650), this portion 
connects to the lower levels of key frame selection at linking point I, and following the process 
through steps 632, and 626, one sees the calculation and use of a second difference metric at step 
624 (Zhang: figure 5B, element 624). Proceeding that step 624 on figure 5B up to linking points 
D up through the flowchart of figure 5A up to linking point C on the initial level dependent upon 
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figure 5, where one sees, the execution of the first difference metric at step 608 (Zhang: figure 5, 
element 608). Accordingly, one can see that Zhang clearly discloses the use of three difference 
measurements. 

For the reasons discussed above, the Examiner maintains the grounds .of rejection. 
Conclusion 

4. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing date 
of this final action. 

5. Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Anand S. Rao whose telephone number is (703)-305-4813. 
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